✅

nit: file a task so we can track and not lose this

✅ #10898

#10579 is now merged!

Can you either: (1) just not sys.exit entirely and let it fail loudly later when it will hit the runtime exception or (2) add a CLI arg to allow skipping this part-- and the part below for the torch.loads

In internal infra, these libraries are loaded a slightly different way and I do not actually pass the .so on command line, and it is not loaded a few lines below.

✅
Ok, so reverted back to our original solution. There is only a warning raised and normally fails later when exporting to ExecuTorch Program:

 # 6. Export to ExecuTorch program
    try:
        exec_prog = edge_program.to_executorch(
            config=ExecutorchBackendConfig(extract_delegate_segments=False)
        )
    except RuntimeError as e:
        if "Missing out variants" in str(e.args[0]):
            raise RuntimeError(
                e.args[0]
                + ".\nThis likely due to an external so library not being loaded. Supply a path to it with the "
                "--portable_lib flag."
            ).with_traceback(e.__traceback__) from None
        else:
            raise e

✅

✅

This probably shouldn't be required because portable library is loaded only when --quantize=True.

✅ Thanks, fixed in latest push.

Is this documentation up to date? Is portable lib built just by specifying these two flags?

The quantized_ops_aot_lib links to portable_lib

$ ldd ./venv3.10/lib/python3.10/site-packages/executorch/kernels/quantized/libquantized_ops_aot_lib.so
        _portable_lib.cpython-310d-x86_64-linux-gnu.so => not found
       ....

For some reason we must load the portable_lib manually prior to libquantized_ops_aot_lib.so, the dlopen does not not find is by its own.

✅
FYI @skywall , we do not need any custom library loading for the quantized kernels out variants. There are already a python packages for this:

import executorch.extension.pybindings.portable_lib
import executorch.kernels.quantized 

Thanks to @digantdesai for the review items which helped me to find it out.

FYI I added optimized cortex-M q/dq int8 op if you want to use that , it is still quite early days for that lib

is this using selective build?

Not sure what you mean.

Ok, I understand where you are heading. We needed the quantized_aot_lib to get the out variants for quantize/dequantize_per_tensor operators.
I find there are already python bindings and modules to solve it:

import executorch.extension.pybindings.portable_lib
import executorch.kernels.quantized 

✅

why do we need these? just include the python module perhaps?

You are right (obviously) , we don't. Importing the python modules instead.

import executorch.extension.pybindings.portable_lib
import executorch.kernels.quantized 

Thanks for the finding, it helped me to locate these python modules.

Nice!

Should we include this in the CI?

I was thinking about it, but as the simulator is not yet ready only reasonable check is if the example not crash and produce some output.

I can include in the CI. No preference here.